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Abstract. We consider probabilistic automata on infinite words with 
acceptance defined by safety, reachability, Buchi, coBiichi, and limit- 
average conditions. We consider quantitative and qualitative decision 
problems. We present extensions and adaptations of proofs for proba- 
bilistic finite automata and present a complete characterization of the 
decidability and undecidability frontier of the quantitative and qualita- 
tive decision problems for probabilistic automata on infinite words. 



1 Introduction 

Probabilistic automata and decision problems. Probabilistic automata 
for finite words were introduced in the seminal work of Rabin [RAB63J, and 
have been extensively studied (see the book by )PAZ71| on probabilistic au- 
tomata and the survey of [BUK80J). Probabilistic automata on infinite words 
have been studied recently in the context of verification [BG05 BBG08J. We 
consider probabilistic automata on infinite words with acceptance defined by 
safety, reachability, Buchi, coBiichi, and limit-average conditions. We consider 
the quantitative and qualitative decision problems [PAZ71IGO09] . The quanti- 
tative decision problems ask, given a rational < A < 1, whether (a) (equality) 
there is a word with acceptance probability exactly A; (b) ( existence ) there is a 
word with acceptance probability greater than A; and (c) (value) for all £ > 0, 
there is a word with acceptance probability greater than A — e. The qualita- 
tive decision problems are the special cases of the quantitative problems with 
A 6 { 0, 1 }. The qualitative and quantitative decision problems for probabilistic 
automata are the generalization of the emptiness and universality problem for 
deterministic automata. 

Known results for probabilistic automata on infinite words. The deci- 
sion problems for probabilistic automata on finite words have been extensively 
studied PAZ71 BUK80]. For probabilistic automata on infinite words it follows 
from the results of [BBG08 that for the coBiichi acceptance condition, the qual- 
itative equality problem is undecidable and the qualitative existence problem 
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is decidable (in EXPTIME), whereas for the Biichi acceptance condition, the 
qualitative equality problem is decidable (in EXPTIME) and the qualitative ex- 
istence problem is undecidable. The qualitative equality and existence problems 
for safety conditions were shown to be PSPACE-complctc in CSV09J, and the 
qualitative equality problem for Biichi conditions and qualitative existence prob- 
lem for coBiichi conditions were shown to be PSPACE-complete in CSV09p . 
The arithmetic hierarchy characterization of the the undecidable problems was 
established in |CSV09|CSV09pl . 

Our results: quantitative decision problems. In GO09 a simple and ele- 
gant proof for the undecidability of the quantitative decision problems for proba- 
bilistic finite automata was given. In Section 3 we show that the proof of (GO09 
can be extended to show that the quantitative decision problems are undecidable 
for safety, reachability, Biichi, coBiichi, and limit-average conditions. In partic- 
ular we show the undecidability of the quantitative equality problem for the 
special classes of absorbing probabilistic automata with safety and reachability 
acceptance conditions. The other undecidability results for quantitative deci- 
sion problems are obtained by simple extensions of the result for quantitative 
equality. 

Our results: qualitative decision problems. In Section 4 we show that all 
qualitative decision problems are decidable for probabilistic safety automata. 
We present a simple adaptation of a proof of [GOCW to give the precise char- 
acterization of the decidability and undecidability frontier for all qualitative 
decision problems for probabilistic reachability, Biichi, and coBiichi automata. 
Concerning probabilistic limit-average automata, we show that the qualitative 
value problem is undecidable. We use the results of |TRA09j to prove that the 
qualitative equality is also undecidable, and that the qualitative existence is 
decidable. 

2 Definitions 

In this section we present definitions for probabilistic automata, notions of ac- 
ceptance for infinite words, and the decision problems. 

2.1 Probabilistic automata 

Probabilistic automata. A probabilistic automaton A is a tuple (Q,q L , £,M.) 
that consists of the following components: 

1. a finite set Q of states and an initial state q L ; 

2. a finite alphabet £; 

3. a set M. = { M a a £ £ } of transition probability matrices M a ; i.e., 
for a £ £ we have M a is a transition probability matrix. In other words, 
for all a £ £ the following conditions hold: (a) for all q, q' £ Q we have 
M a (q,q') > 0; and (b) for all q £ Q we have J2 q >eQ m <t(q, q') = 1- 



Infinite paths and words. Given a probabilistic automaton A, an infinite path 
7r = (go, Qi, <l2, ■ ■ •) is an infinite sequence of states such that for all i > 0, there 
exists a G £ such that M a (qi,qi + i) > 0. We denote by n a path in .4, and by 
77 the set of all paths. For a path n, we denote by Inf (it) the set of states that 
appear infinitely often in tt. An infinite (resp. finite) word is an infinite (resp. 
finite) sequence of letters from S. For a finite word w we denote by \w\ the 
length of w. Given a finite or an infinite word to, we denote Wi as the i-th letter 
of the word (for a finite word to we assume i < \w\). 

Cones and probability measure. Given a probabilistic automaton A and a 
finite sequence q = (q , q\, . . . , q n ) of states, the set Cone(g) consists of the set 
of paths 7r with prefix q. Given a word to G , we first define a measure fi w on 
cones as follows: 

1. [i w (Cone(q L )) — 1 and for q^ ^ q u we have \x w (Cone(gj)) = 0; 

2. for a sequence q = (go, qi, . . . , q n -i,q n ) we have 

^(Cone^)) = y u u, (Cone(g ,'Zi, • • • ,<?n-i)) ■ M Wn (q n _ 1 ,q n ). 

The probability measure is the unique extension of the measure \i w to the 
set of all measurable paths in A. 

2.2 Acceptance conditions 

Probabilistic automata on finite words. A probabilistic finite automaton 
consists of a probabilistic automaton and a set F of final states. The automaton 
runs over finite words to e U* and for a finite word to = oqO\ . . . <t„ G S* it 
defines a probability distribution over Q as follows: let o"o(g t ) = 1 and for t > 
we have d~i + i = Si ■ M ai . The acceptance probability for the word to, denoted as 
Va(w), is E 9 gf S \w |+i(9) • 

Acceptance for infinite words. Let .4 be a probabilistic automaton and let 
F C Q be a set of accepting (or target) states. Then we consider the following 
functions to assign values to paths. 

1. Safety condition. The safety condition Safe(.F) defines the set of paths in A 
that only visits states in F; i.e., Safc(F) = { (qo,qi, ■ ■ ■) \ Vt > 0. qi G F }. 

2. Reachability condition. The reachability condition Rcach(F) defines the 
set of paths in A that visits states in F at least once; i.e., Reach(F) = 
{(q 0l gi ,...) I 3t > 0. q t EF}. 

3. Biichi condition. The Biichi condition Biichi(_F) defines the set of paths in A 
that visits states in F infinitely often; i.e., Biichi(F) = {tt \ Inf(7r)n-F 7^ 0}. 

4. coBiichi condition. The coBiichi condition coBuchi(.F) defines the set of 
paths in A that visits states outside F finitely often; i.e., coBiichi(F) = 
{tt I Inf(Tr) C F}. 

5. Limit- average condition. The limit-average condition is a function 
LimitAvg : 77 — > IR that assigns to a path the long-run average frequency of 



the accepting states. Formally, for a state q <G Q, let r(q) = 1 if q G F and 
otherwise, then for a path it = (q , q\, . . .) we have 



In sequel, we will consider Reach, Safe, Biichi, coBiichi and LimitAvg as functions 
from 77 to H. Other than LimitAvg, all the other functions only returns boolean 
values (0 or 1). Given an condition <P : U — > H, a probabilistic automaton A 
and a word w, we denote by E^(#) the expectation of the function <P under the 
probability measure P^. Given a probabilistic automaton A and a condition 
we use the following notation: Ai'P, w) — E^(^>), and if the condition is clear 
from the context we simply write A(w). If <P is boolean, then A(<P, w) is the 
acceptance probability for the word w for the condition # in A. 

2.3 Decision problems 

We now consider the quantitative and qualitative decision problems. 

Quantitative decision problems. Given a probabilistic automaton A, a con- 
dition and a rational number < A < 1, we consider the following questions: 

1. Quantitative equality problem. Does there exist a word w such that E^(<£) — 
X. If is boolean, then the question is whether there exists a word w such 
that P^(<7>) = A. 

2. Quantitative existence problem. Does there exist a word w such that E^J(^) > 
A. If <fr is boolean, then the question is whether there exists a word w such 
that P^(^) > A. This question is related to emptiness of probabilistic au- 
tomata: let Ca^i A) = {w | P^(^) > A} be the set of words with acceptance 
probability greater than A; then the set is non-empty iff the answer to the 
quantitative existence problem is yes. 

3. Quantitative value problem. Whether the supremum of the values for all 
words is greater than A, i.e., sup^g^E^^) > A. If <P is boolean, this 
question is equivalent to whether for all £ > 0, does there exist a word w 
such that P^(<7>) > A-e. 

Qualitative decision problems. Given a probabilistic automaton A, a condi- 
tion <£, we consider the following questions: 

1. Almost problem. Does there exist a word w such that E^J(^) = 1. 

2. Positive problem. Does there exist a word w such that E^(<?) > 0. 

3. Limit problem. For all e > 0, does there exist a word w such that E^(<?) > 




fe-i 



i=0 



1 -£. 



If <P is boolean, then in all the above questions E is replaced by P. 



3 Undecidability of Quantitative Decision Problems 



In this section we study the computability of the quantitative decision problems. 
We show undecidability results for special classes of probabilistic automata for 
safety and reachability conditions, and all other results are derived from the 
results on these special classes. The special classes are related to the notion of 
absorption in probabilistic automata. 

3.1 Absorption in probabilistic automata 

We will consider several special cases with absorption condition and consider 
some simple equivalences. 

Absorbing states. Given a probabilistic automaton A, a state q is absorbing if 
for all a € £ we have M a (q, q) = 1 (hence for all q' ^ q we have M a (q, q') = 0). 

Acceptance absorbing automata. Given a probabilistic automaton A let F 
be the set of accepting states. The automaton is acceptance-absorbing if all states 
in F are absorbing. Given an acceptance-absorbing automaton, the following 
equivalence hold: Reach(F) = Biich^f 1 ) = coBuchi(F) = LimitAvg(-F). Hence 
our goal would be to show hardness (undecidability) for acceptance-absorbing au- 
tomata with reachability condition, and the results will follow for Biichi, coBiichi 
and limit-average conditions. 

Absorbing automata. A probabilistic automaton A is absorbing if the follow- 
ing condition holds: let C be the set of absorbing states in A, then for all a E S 
and for all q G (Q \ C) we have M a (q, q') > for some q' e C. In sequel we will 
use C for absorbing states in a probabilistic automaton. 

3.2 Absorbing safety automata 

In sequel we write automata (resp. automaton) to denote probabilistic automata 
(resp. probabilistic automaton) unless mentioned otherwise. We now prove some 
simple properties of absorbing automata with safety condition. 

Lemma 1. Let A be an absorbing automaton with C as the set of absorbing 
states. Then for all words w G we have ¥^{Reach(C)) = 1. 

Proof. Let -q = min 9e Q i<T6 £ X) g 'ec M a (q, q') be the minimum transition proba- 
bility to absorbing states. Since A is absorbing we have r\ > 0. Hence for any 
word w, the probability to reach C after n steps is at least 1 — (1 — r/) n . Since 
r\ > 0, as n — > oo we have 1 — (1 — rf) n 1, and hence the result follows. I 

Complementation of absorbing safety automata. Let A be an absorbing 
automaton, with the set F as accepting states, and we consider the safety con- 
dition Safe(i 7 '). Without loss of generality we assume that every state in Q \ F 
is absorbing. Otherwise, if a state in q e Q \ F is non-absorbing, we transform 
it to an absorbing state and obtain an automaton A'. It is easy to show that for 



the safety condition Safe(F) the automata A and A' coincide (i.e., for all words 
w we have A(Sa£e(F),w) = A' (Saie(F) , w)) . Hence we consider an absorbing 
safety automaton such that all states in Q \ F are absorbing: it follows that 
every non-absorbing state is an accepting state, i.e., Q \C C F. We claim that 
for all words w we have 

P^(Safe(F)) = PJ(Reach(F n C*)). 

Since every state in Q \ C Cf and all states in C are absorbing, it follows that 
P^(Safe(F)) > P^(Reach(TnC)). Since A is absorbing, for all words w we have 
P^(Reach(C)) = 1 and hence it follows that 

F£(Safe(F)) < P^(Reach(C)) • P^(Safe(F) | Reach(C)) = P^(Reach(F n C)). 

The complement automaton A is obtained from A by changing the set of ac- 
cepting states as follows: the set of accepting states F in A is (Q \ C) U (C\ F), 
i.e., all non-absorbing states are accepting states, and for absorbing states the 
accepting states are switched. Since A is also absorbing it follows that for all 
words w we have PH-(Safe(F)) = P^-(Rcach(C n F)) = PH.(Reach(C \ F)). Since 
A is absorbing, it follows that for all words we have P^-(Reach(C)) = 1 and hence 
P^(Safe(F)) =P^-(Reach(C\F)) = 1 -P^(Reach(CnF)) = 1 -P^(Reach(Cn 
F)). It follows that for all words w we have .4(Safe(F), w) = 1 - _4(Safe(F), w), 
i.e., A is the complement of A. 

Remark 1. It follows from above that an absorbing automaton with safety condi- 
tion can be transformed to an acceptance-absorbing automaton with reachabil- 
ity condition. So any hardness result for absorbing safety automata also gives a 
hardness result for acceptance-absorbing reachability automata (and hence also 
for Buchi, coBiichi, limit-average automata). 

3.3 Undecidability for safety and acceptance-absorbing reachability 
automata 

Our first goal is to show that the quantitative equality problem is undecid- 
able for safety and acceptance-absorbing reachability automata. The reduction 
is from the Post Correspondence Problem (PCP). Our proof is inspired by and 
is an extension of a simple elegant proof of undecidability for probabilistic finite 
automata [GO09I . We first define the PCP and some related notations. 

PCP. Let ipi,tf2 : S — > { 0, 1, . . . , k — 1 }*, and extended naturally to S* 
(where k = The PCP asks whether there is a word w G S* such that 

tp^w) = ip 2 (w). The PCP is undecidable [HU79] , 

Notations. Let ip : { 0, 1, . . . , k — 1 }* — > [0, 1] be the function defined as: 

CT„ (T„_i 0~2 o~i 

^{a l0 - 2 ...a n ) = — + — + ■■■ + — T + — . 



For i e { 1, 2 }, let 0, = ip otfi : S* — > [0, 1]. Wc first prove a property of 9i, that 
can be derived from the results of |BMT77j . 




Fig. 1. Absorbing safety automata from PCP instances. 



Lemma 2. For a finite word w e S* and a letter a e E we have 
9i{wo-) = 6i{o-)+6i{w)-ki(o-), 

where k,(a) = AH^MI . 

Proof. Let w = o~\o 2 . . . cr n , and let <pi(w) — b\b 2 • • • b m and <fi(cr) = a\a 2 ■ ■ ■ ag. 
Then we have 

6i(w ■ a) — ip o (fi(w • cr) 

= ip(hb 2 . . .b m a\a 2 ...at) 

at ai b m bi 
= — -I 1 — - -\ — -| 1 _ 

k k l k t+1 fci+m 

U + W k* \k + + fc™ j 

= 1p o ^(ct) + p • (ip o ^(w)) 
= ^(a)+^H -ki(a). 

The result follows. I 

The absorbing safety automata from PCP instances. Given an instance 
of the PCP problem we create two safety automata Ai = (Q, SLl{%}, M l , q L , F), 
for i S { 1, 2 } as follows (we assume $ ^ £): 

1. fSei 0/ states and initial state). Q = { qo, q±, q 2 , 93 } and g t = qo; 

2. (Accepting states). F = {q , g 1; c/ 3 }; 

3. (Transition matrices). The set of transition matrices is as follows: 

(a) the states q 2 and c/3 are absorbing; 

(b) Mi(q ,q 2 ) = 1 and Afjfai,©) = 1; 

(c) for all cr G Z 1 we have 

i. M*(qo,q 2 )=M*(q 1 ,q 2 ) = \\ 

ii. M< ( M ) = ± • (1 - W); M*(g , «i) = \ ■ W; 

hi. M^( qi ,q ) = \ ■ (1 -^(a) - fci(cr)); M*(gi,gi) = ± • (^(a) + fc^a)); 



A pictorial description is shown in Fig [TJ We will use the following notations: 
(a) we use E for E U { $ }; (b) for a word w G E u if the word contains a $, then 
we denote by first(iu) the prefix iv' of w that does not contain a $ and the first $ 
in w occurs immediately after w' (i.e., w'$ is a prefix of w). We now prove some 
properties of the automata A4. 

1. The automata Ai is absorbing: since for every state and every letter the 
transition probability to the set { 52 , 93 } is at least i ; and g 2 and q$ are 
absorbing. 

2. Consider a word w G E u . If the word contains no $, then the state qi is 
reached with probability 1 (as every round there is at least probability i 
to reach g 2 ). If the word w contains a $, then as soon as the input letter is 
$, then the set { 921 93 } is reached with probability 1. Hence the following 
assertion holds: the probability P^(Safe(-F)) is the probability that after the 
word w' — first(ui) the current state is q\. 

Lemma 3. For all words w G E* , the probability that in the automaton Ai after 
reading w (a) the current state is q\ is equal to -9i(w); (b) the current state 

is qo is equal to ^fbr ' (1 — $i( w ));' an d ( c ) the current state is q 2 is equal to 

I l 

L 2RT 

Proof. The result follows from induction on length of w, and the base case is 
trivial. We prove the inductive case. Consider a word w ■ a: by inductive hypoth- 
esis the probability that after reading w the current state is go is • (1 — 9i{w)) 
and the current state is gi is ^fbr ' 9i{ w )i an d the current state is (72 with proba- 
bility (1 — 2~Fh)- After reading a, if the current state is go or gi, with probability 
i a transition to g2 is made. Hence the probability to be at g2 after w ■ a is 
(1 — 9M+1 ) an d the rest of the probability is to be at cither go and q\. The 
probability to be at gi is 

^i+r • ((1 - Oi{w)) ■ 9i(a) + B^w) ■ (9 t (a) + k t (a)) = ^_ • (#» + B^w) ■ h(a)) 

The first equality follows by rearranging and the second equality follows from 
Lemma [2] Hence the result follows. I 

The following lemma is an easy consequence. 

Lemma 4. For i G { 1,2}, for a word w G E^ , (a) if w contains no $, then 
Ai(w) = 0; (b) if w contains a $, let w' = first(w) 7 then Ai(w) — • 9i(w'). 

Constant automata and random choice automata. For any rational con- 
stant v it is easy to construct an absorbing safety automaton A that assigns 
value v to all words. Given two absorbing safety automata A\ and A 2 , and two 
non- negative rational numbers Pi, 02 such that Pi + (3 2 = 1, it is easy to con- 
struct an automaton A (by adding an initial state with initial randomization) 
such that for all words w we have A(w) = [3\ ■ A\{w) + (3 2 ■ A 2 (w). We will use 
the notation /3i ■ A\ + 02 • A 2 for the automaton A. 




Fig. 2. Safety automaton As- 



Safety automaton A3. We consider a safety automaton A3 on the alphabet 
S as follows: (a) for a word w without any $ the acceptance probability is 1; 
(b) for a word with a $ the acceptance probability is 0. The automaton is shown 
in Fig [5] and the only accepting state is qo. Consider the acceptance-absorbing 
reachability automaton A4 as follows: the automaton is same as in Fig [5] with 
accepting state as q%: the automaton accepts a word with a $ with probability 1, 
and for words with no $ the acceptance probability is 0. 

Theorem 1 (Quantitative equality problem). The quantitative equality 
problem is undecidable for probabilistic safety and acceptance-absorbing reach- 
ability automata. 

Proof. Consider an automaton A — | • Ai + | • (1 — A2) + § • A3 (since A2 is 
absorbing we can complement A2 and obtain an automaton for 1— ^4.2)- We show 
that the quantitative equality problem for A with A = | is yes iff the answer to 
the PCP problem instance is yes. We prove the following two cases. 

1. Suppose there is a finite word w such that ipi(w) — (pzfa). Consider the 

infinite word w* = w$w' where w' is an arbitrary infinite word. Then the 

acceptance probability of w* in A3 is (since it contains a $). Since w* 

contains a $ and first(w*) = w, by Lemma HI the acceptance probability of 

w* in A is I + ■ {0\{w) — 9 2 {w)). Since ipi(w) — <P2(w) it follows that 

8i(w) — 9%{w) = 0. It follows that the acceptance probability of w* in A is 
1 

3" 

2. Consider an infinite word w. If the word contains no $, then the acceptance 
probability of w in Ai and A2 is 0, and the acceptance probability in ^3 is 1. 
Hence A accepts w with probability | > i. If the word w contains a $, then 
.A3 accepts w with probability 0. The difference of acceptance probability 
of w in A\ and A 2 is • (6>i(u>') — 9 2 {w')), where w' — first (w). Hence 
the acceptance probability of w in A is § iff 6»i (w') = e 2 (w'). Hence w' is a 
witness that ipi(w') = ip 2 (w'). 

It follows that there exists a finite word w that is a witness to the PCP instance 
iff there exists an infinite word w* in A with acceptance probability equal to ^. 

For acceptance-absorbing reachability automata: consider the same construc- 
tion as above with A3 being replaced by A4 , and the equality question for A = | . 
Since A% — A2 < 1, it follows that any witness word must contain a $, and then 
the proof is similar as above. The result for acceptance-absorbing reachability 
automata also follows easily from |BMT77j .l 



1 



Fig. 3. Safety automaton A$. 



Quantitative existence and value problems. We will now show that the 
quantitative existence and the quantitative value problems are undecidable for 
probabilistic absorbing safety automata. We start with a technical lemma. 

Lemma 5. Let us consider a PCP instance. Let z = max ffe s{ |<pi(er)|, | ^2 (c) | }■ 
For a finite word w, if 8 1 (w) — 9 2 {w) ^ 0, then (9i(w) — 02{w)) 2 > k2 .\w\.* ■ 

Proof. Given the word w, we have |<pi(u>)| < \w\ ■ z, for i G { 1,2 }. It follows 
that 9i(w) can be expressed as a rational number 2i, where q < k z '' w ^. It follows 
that if 6i(w) ^ 9 2 {w), then \6i(w) - 9 2 (w)\ > The result follows. I 

Automaton A5. Consider the automaton shown in Fig [3] the accepting states 
are go and q%. For any input letter in S, from the initial state qo the next state 
is itself with probability fc2 -^+i) 1 an d the rest of the probability is to goto q 2 . For 
input letter $ the next state is q\ with probability 1. The states q\ and q 2 are 
absorbing. Given a word w, if there is no $ in w, then the acceptance probability 
is 0; otherwise the acceptance probability is fc2 (*+i)-| m 'i 1 where w' — first(w). Also 
note that the automaton A§ is absorbing. 

Theorem 2 (Quantitative existence and value problems). The quanti- 
tative existence and value problems are undecidable for probabilistic absorbing 
safety automata. 

Proof. Let us first consider the following automata: B\ = \ ■ A\ + 5 • (1 — A%) 
and £>2 = i • A2 + 5 ■ (1 — Ai). The Cartesian product of B\ and B 2 gives us 
the automaton B3 = j — (Ai — A2) 2 . Finally, let us consider the automaton 
B = I • £>3 + I • A5. The automaton B can be obtained as an absorbing safety 
automaton. We show that there exists a word (or the sup value of words) is 
greater than | iff the answer to PCP is yes. 

1. If the answer to PCP problem is yes, consider the witness finite word w 
such that tfii(w) — tf2(w). We construct an infinite word w* for B as follows: 
w* — w$w' where w' is an arbitrary infinite word. Since the word w* contains 
a $ by Lemma |4] we have the difference in acceptance probability of w* in 
A\ and A2 is ■ {9\{w) — Q^w)). Since tfii(w) = (p 2 (w), it follows that 
Ai{w*) = Aaiw*). Hence we have B(w*) = | + fc2 . (z _ l 1 1) . |m > |. 



2. We now show that if there is an infinite word with acceptance probabil- 
ity greater than | (or the sup over the infinite words of the acceptance 
probability is greater than g) in £>, then the answer to the PCP prob- 
lem is yes. Consider an infinite word w for B. If w contains no $, then 
A\, A 2 , and ^5 accepts w with probability 0, and hence the acceptance 
probability in B is g. Consider an infinite word w that contains a $. Let 
w' = first(w). If 9i(w') ^ Oi{w'), then by Lemma|l]and Lemma[5]we have 

(Ai(w) - A 2 (w)) 2 > -p-^rr > ■p-RT^+TT- Since A ?>( w ) = F-RT^+TT' 
it follows that B{w) < 1/8. If 9i(w') = 6 2 (w') (which implies tpi(w') = 
ip 2 (w')), then B{w) = \ + fc2 . |m /}. ( , +1) > |. 

It follows from above that the quantitative existence and the quantitative value 
problems are undecidable for absorbing safety (and hence also for acceptance- 
absorbing reachability) automata. I 

Corollary 1. Given any rational < A < 1, the quantitative equality, exis- 
tence, and value problems are undecidable for probabilistic absorbing safety and 
acceptance-absorbing reachability automata. 

Proof. We have shown in Theorem Q] and Theorem [2] that the problems are 
undecidable for specific constants (e.g., |, i etc.). We show below given the 
problem is undecidable for a constant < c < 1 , the problem is also undecidable 
for any given rational < A < 1. Given an automaton B we consider two cases: 

1. If A < c, then consider the automaton A — — -B+(l — -)■(). The quantitative 
decision problems for constant c in B is exactly same as the quantitative 
decision problems for A with A. 

2. If A > c, then consider the automaton A — jE^-B+ ■ 1. The quantitative 
decision problems for c in B is exactly same as the quantitative decision 
problems for A with A. 

The desired result follows. The result for acceptance- absorbing reachability au- 
tomata also follows from [BMT77] . I 

Corollary 2 (Quantitative decision problems). Given any rational < 
A < 1, the quantitative equality, existence and value problems are undecidable 
for probabilistic safety, reachability, Biichi, coBiichi and limit-average automata. 

4 (Un-) Decidability of Qualitative Decision Problems 

In this section we show that the qualitative decision problems are decidable for 
safety automata, and the limit problem is undecidable for reachability, Biichi, 
coBiichi and limit-average automata. The result is inspired from the proof 
of |GO09j that shows that the limit problem is undecidable for finite automata. 
If the finite automata construction of GO 09] were acceptance absorbing, the 
result would have followed for acceptance-absorbing reachability automata (and 



hence also for Biichi, coBiichi and limit-average automata). However, the un- 
decidability proof of [GO09| for finite automata constructs automata that are 
not acceptance-absorbing. Simply changing the accepting states of the automata 
constructed in [GO 09 to absorbing states does not yield the undecidability for 
accepting absorbing automata. We show that the construction of GO09] can be 
adapted to prove undecidability of the limit problem for acceptance-absorbing 
automata with reachability condition. We first present a simple proof that for 
safety condition the almost and limit problem coincide. 

Lemma 6. Given an automaton with a safety condition the answer to the limit 
problem is yes iff the answer to the almost problem is yes. 

Proof. If the answer to the almost problem is yes, then trivially the answer to the 
limit problem is yes (since a witness for almost problem is a witness for all e > 
for the limit problem). We now show the converse is true. Consider an automaton 
A with I states, and let r\ > be the minimum non-zero transition probability 
in A. We assume that the answer to the limit problem is yes and show that the 
answer to the almost problem is also yes. Consider e = rj 2 , and let w be a word 
such that A(w) > 1 — e. For a position i of w let Si = {q € Q | <W|+i(<z) > 0} be 
the set of states that have positive probability in the distribution of states after 
reading the prefix of length i oiw. If a path in A leaves the set F of accepting 
states within k steps, then it leaves with probability at least r] k . Since w ensures 
the safety condition with probability at least 1 — e = 1 — rj 2 it follows that for 
all 1 < i < 2 e we have Si C F (i.e., each Si upto 2 e is a subset of the accepting 
states). There must exist < n < m < 2 e such that S n = S m . Consider the 
word w* = wiu>2 ■ • ■ w n -i{w n w n +iw m ^i) u . The word w* ensures the following: 
let S* — { q <E Q | £| m ;|+i(gO > }, then for all < i < n we have Si = 5*, and 
for alii > n we have S* C S t . It follows that for all i > we have S* C F 
and hence A(w*) — 1. The result follows. I 

Qualitative decision problems for safety conditions. The above lemma 
shows that for safety condition the answers to the almost and the limit problem 
coincide. The almost and positive problems for safety condition are PSPACE- 
complctc CSV09 . It follows that the qualitative decision problems are decidable 
for safety condition. 

Lemma 7. Consider the acceptance-absorbing automaton A = \A\ + \ ■ A2, 
where A\ and A2 are shown in Fig^ The following assertion hold: for all e > 0, 
there exists a word w such that A(w) > 1 — e iff x > i. 

Proof. Given e > 0, a word w to be accepted with probability at least 1 — | 
both Ai and A2 must accept it with probability at least 1 — e. Conversely, given 
a word w if it is accepted by Ai and A2 with probability at least then A 

accepts it with probability at least 1 — Hence we show that for all e > there 
exist words w such that both Ai(w) > 1 — e and Az(w) > 1 — e iff x > |. 

We first observe that a word w to be accepted in Ai can have a $ only 
after having reached its absorbing accepting state (since for all other states the 
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Fig. 4. Automata A\ (on left) and .A2 (on right). 



input letter $ leads to the non-accepting absorbing state q^). A word w to be 
accepted in A2 must have a S, and the $ must occur when the current state is 
either (75 or q$. A word to be accepted in A\ must contain a b and the b must 
occur when the current state is q\. Hence it suffices to consider words of the 
form w* — ww' ', where w' is an arbitrary infinite word and w is a finite word of 
the form a no ba ni b. . .a ni b$. Consider a word a n b, for n > 0: the probability to 
reach the state (73 from q\ is x n , and the probability to reach the absorbing non- 
accepting state qr from q$ is (1 — x) n . Consider a word w = a n °ba ni b . . . a ni b$: 
(a) the probability of reaching q$ from qi in Ai is 1 — 111=0 (1 — £ ni ); (b) the 
probability of reaching (77 from q§ in .A2 is 

(1 - x) ni + (1 - (1 - x)™ 1 ) • (1 - x) n ^+ ■ . ■ +ni= 1 ( 1 - c 1 - a; )™ fc ) • (i - x ) n ' 

= 1 -nLoa-a -*)-)■ 

Ex < |, then a: < 1-x, and hence 1 - IlLo C 1 ~ ^ ) ^ 1 "JTk^oi 1 - i 1 ~ X T')- 
It follows that the acceptance probability of w in _4i is less than the rejection 
probability in A%, hence Aiw) < 5. It follows that if x < |, then for all words 
w we have A(w) < |. 

To complete the proof we show that if x > |, then for all e > 0, there exist 
words u> such that both Ai(w) > 1 — £ and Aq(w) > 1 — e. Our witness words 
w* will be of the following form: let w = a n °ba ni b . . . a ni b and we will have 
w* = Our witness construction closely follows the result of |GO09j . Given 
the word w, the probability to reach q% from q\ is 

i 

L 1 = i-l[(i-x n *y, 

k=0 



hence Ai accepts w* with probability L\. Given the word w, since the last letter 
is a b, with probability 1 the current state is either 57 or q 5 . The probability to 
reach from q 5 for w is given by 



l 2 = (1 - z)" 1 + (1 - (1 - x)™ 1 ) • (1 - X ) n * + ■■■+ nLU 1 - (i - x ) nk ) ■ (i - 

< EL=o(! - since Ili=o(l - (1 - ^)" fc ) < 1 ^ all J < »• 

Thus the acceptance probability for w in A2 is at least 1 — L 2 . Hence given e > 0, 
our goal is to construct a sequence (no, n\, n 2 , ■ ■ ■ , n») such that £1 > 1 — e and 
L 2 < e. For e > 0, it suffices to construct an infinite sequence (rik)k&i such that 

JJ(l-x) n * =0; and ^(1 - x) n " < e. 
fc>0 fc>o 

Then we can construct a finite sequence (no, ni, . . . , rij) of numbers such that 
we have both 

i i 

Y[(l ~x) nk < e; and ^(1 - a;)" fc < 2 • e, 

fe=0 k=0 

as desired. Since Ofe>o(^ — = iff Y^k=o x1lk = 00 ! we construct sequence 
(n fc ) feeN such that X^fclo x " fc = °°> anc l Sfe>o( 1 - x ) Uk - £ - Let ( n fc)feeN be a 
sequence such that n k = ln x (i) + where J is a suitable constant (to be chosen 
later and depends on e and x). Then we have 

k>0 fe>0 

On the other hand, we have 

Since x > \ we have /? = '"^"^ > 1, i.e., there exists (3 > 1 such that l — x^x 13 . 
Hence 

2(i - xy- = 2 ^'" fc = xfj ' J ■ E x0 ' 1Mi) = x "' J • E p- 

fc>0 fc>0 fe>0 fe>0 

Since the above series converges, for all e > 0, there exists a J such that ^2 k>0 (l- — 
x) nk < e. This completes the proof. I 

Almost and limit problem do not coincide. In contrast to probabilistic 
safety automata where the almost and limit question coincide, the answers are 
different for probabilistic acceptance-absorbing reachability automata. In the 
automaton A above if x > i, the answer to the limit question is yes. It follows 
from the proof above that if 5 < x < 1, then though the answer to the limit 
question is yes, the answer to the almost question is no. The almost and positive 



problem for reachability automata is decidable by reduction to POMDPs with 
reachability objective. We show that the limit question is undecidable. The proof 
uses the above lemma and the reduction technique of |GOQ9j . 

Reduction for undecidability. Given an acceptance-absorbing reachability 
automaton B, we construct an automaton as shown in Fig [3] we assume that (j 
does not belong to the alphabet of B, and in the picture the dashed transitions 
from B on (| are from accepting states, and the solid transitions are from non- 
accepting states. We call the automaton as A*. We claim the following: the 
answer to the limit problem for the automaton A* is yes iff there exists a word 
w that is accepted in B with probability greater than i. 

1. Suppose there is a word w* such that B(w*) = y > h. Let rj = y — ^. There 
is a finite prefix w of w* such that the probability to reach an accepting state 
in B given w is x = y — ^ > ^. Given e > and x as defined y — 3 , we have 
x > i, and hence by Lemma [7] there exists a sequence (no, ni, . . . , n^) such 
that (a n °ba ni b. . . a nk b)($) u ' is accepted in A with probability at least 1— e. It 
follows that the automaton A* accepts ((aw$) n °b(aw§) ni . . . b{aw^) rik b){%) u: 
with probability at least 1 — e. It follows that the answer to the limit problem 
is yes. 

2. If for all words w we have B(w) < Y2, then we show that the answer to 
the limit problem is no. Let x — snp wGS u, B{w) < \. Consider a word w = 
((aw $) n °b{aw4) n i . . . b(aw 2 i) nk b){$) a ': given this word the probability to 
reach (73 in q\ given w is at most 

i 

X-Ylil-x"*); 

fe=0 

and the probability to reach qy from q§ given w is at least 

i-l 

(l-x) ni + (1 - (1 - .x)" 1 ) • (l-x) n2 + ■■■ + - (l-x) nk ) ■ (l-a;) ni 

k=0 

i 

= i-U(i-(i-xD. 

k=0 

The argument is similar to the proof of Lemma [7] It follows from the argu- 
ment of Lemma [7J that the word is accepted with probability at most h in .4*. 
Thus it follows that for all words w we have A* (w) < i . Since the quantita- 
tive existence problem is undecidable for acceptance absorbing reachability 
automata (Theorem^, it follows that the limit problem is also undecidable. 

Undecidability. From the above proof it follows that the limit problem is un- 
decidable for acceptance absorbing reachability automata, and hence also for 
Biichi, coBiichi, and limit-average condition. 

The case of Limit-average Automata. The proofs for the decidability and 
undecidability results concerning limit-average conditions differ from the other 
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Fig. 5. Limit undecidability construction. 
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Dec. 


Undec. 


Reachability 


Dec. 


Dec. 


Undec. 


Undec. 


Biichi 


Undec. 


Dec. 


Undec. 


Undec. 


coBiichi 


Dec. 


Undec. 


Undec. 


Undec. 


Limit-average 


Dec. 


Undec. 


Undec. 


Undec. 



Table 1. Decidability and undecidability results for probabilistic automata. 

conditions. We present how to use the results of [TRA09] and [TRAj to prove 
that the positive problem for limit-average automata is PSPACE-complete, and 
the almost problem for limit-average automata is undecidable. Given n G Ft, the 
authors of [TRA09j define: 

Supp(7r) = {q e Q s.t. LimitAvg(7r) > 0} 

In paper [TRA09] and [TRAj are considered the following decision problems: 

— Positive Strong recurrence Problem: Given a PA A, decide whether 
there exists i«e£ u such that F^[{tt e U\ Supp(7r) n F ^ 0}] > 0. 

— Almost Sure Strong co-recurrence Problem: Given a PA A, decide 
whether there exists w e such that P^[{tt E II | Supp(7r) n F = 0}] = 1. 

The first point of the following theorem was shown in [TPS] , and the second 
in [TRA09] . an extended version of |TRA] . 

Theorem 3. The Positive Strong recurrence Problem is PSPACE-complete, and 
the Almost Sure Strong co-recurrence Problem is undecidable. 



By definition, for any PA A and any w £ for all it S 77, we have 
Supp(7r) n 7^ iff LimitAvg(7r) > 0. Also, for any PA A and any w £ S u , for 
all 7r G 77, we have that Supp(7r) n F = iff Supp(7r) C F, iff Limit Avg(7r) = f . 
Finally, we get: 

— The following problem is PSPACE complete: given a PA A, decide whether 
there exist w £ E u such that P^[{tt | LimitAvg(?r) > = 0}] > 0. 

— The following problem is undecidable: given a PA A, decide whether there 
exist such that P^[{tt | LimitAvg(Tr) = f }] = 1 . 

Corollary 3. The positive problem for limit- average automata is PSPACE- 
complete, and the almost problem for limit-average automata is undecidable 

Proof. Concerning the first point, a standard result of probability theory says 
that on a probability space (12, J 7 , /x), given a function / : Q — > K + , we have: 

W\f \ > iff /i({vr £ Q s.t. f(x) > 0}) > 

Given w £ E u , taking fl the set of paths 77, T as the cr-algebra generated by 
cones on 17, /i as and / as the Limit Avg function, we get that: 

E^[LimitAgv] > iff P^[{tt e 77 s.t. Limit Avg(7r) > 0}] > 

Hence the Positive Limit-average Problem is equivalent to the Positive Strong 
recurrence Problem, which is PSPACE-complete. 

Let us now consider the second point. Let LimitAvg : 77 — > {0, 1} be such 
that for all ir G 77 we have LimitAvg(7r) = f — LimitAvg(7r). By the same 
argument as before, given w £ S u , we have: 

[LimitAvg] > iff P^[{tt £ 77 s.t. LimitAvg(Tr) > 0}] > (f) 

Now, [LimitAvg] = f — [Limit Avg] . As a consequence, we have that 
E^ [LimitAvg] = 1 iff E^ [LimitAvg] = 0, hence by equation Q] 

E^ [LimitAvg] = f iff P^[{tt £ 77 s.t. LimitAvg(Tr) > 0}] = 

Which is also equivalent to P^[{7r £ II s.t. Limit Avg(7r) = 0}] = 1, and to 
P^([{7r £ 77 s.t. LimitAvg("7r) = f}] = f. The remark after theorem [3] complete 
then the proof. 

Summary of results. We have seen that the limit problem is undecidable for 
acceptance absorbing reachability automata, and hence also for Biichi, coBuchi, 
and limit-average condition. 

This gives us the Theorem |U For other qualitative questions the results are 
as follows: (A) We argued that for safety condition that almost and limit prob- 
lem coincide, and the decision problems are PSPACE-complete CSV09 . (B) For 
reachability condition the almost and positive problem can be answered through 
POMDPs ( |BBG08j ). and the almost problem was shown to be PSPACE- 
complete [CSV09] . (C) For Biichi condition it was shown in [BBG08j that the 



positive problem is undecidable and the almost problem is decidable in EXP- 
TIME. The almost problem was later shown to be PSPACE-complete |CSV09p| . 
(D) For coBiichi condition it was shown in [BB G08] that the almost problem is 
undecidable and the positive problem is decidable through solution of POMDPs 
with coBiichi condition in EXPTIME [BGG09 . The positive problem was later 
shown to be PSPACE-complete in |CSV09p] . (E) For limit-average condition 
the results follow from the last paragraph. For the exact arithmetic hierarchy 
characterization of the undecidable problems of BBG08 see CSV09 CSV09p . 
The results are summarized in Table [T] (the results for quantitative existence and 
quantitative value problem is the same as for quantitative equality problem). 

Theorem 4 (Qualitative problem). The following assertions hold: (a) the 
positive problem is decidable for probabilistic safety, reachability, coBiichi and 
limit-average automata, and is undecidable for probabilistic Biichi automata; 
(b) the almost problem is decidable for probabilistic safety, reachability, and 
Biichi automata, and is undecidable for probabilistic coBiichi and limit-average 
automata; and (c) the limit problem is decidable for probabilistic safety automata 
and is undecidable for probabilistic reachability, Biichi, coBiichi and limit-average 
automata. 
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